Techniques in which shape models of work objects are provided to a robot in advance so that the robot recognizes a work object existing in the working space and automatically grasps the work object have been studied and developed (for example, see Patent document 4). However, although such model-based techniques can be implemented in working spaces in which the types of work objects are limited, it is substantially impossible to apply them to real environments with complex conditions such as ordinary living environments.
Meanwhile, techniques in which an operator teaches actions to a robot have been known (for example, see Patent documents 1 to 3). Patent documents 1 to 3 disclose techniques in which an operator teaches actions to a robot by using an action teaching apparatus with a graphic display function.
The invention disclosed in Patent document 1 has been made by the inventors of the present application. That invention relates to an action teaching apparatus that teaches actions in a 3D (three-dimensional) space to a robot by using an action teaching apparatus with a 2D (two-dimensional) display. This action teaching apparatus is applicable to a situation where a robot grasps a work object whose shape and 3D position in the working space are unknown. Teaching to a robot by using this action teaching apparatus is performed in the following manner.
Firstly, a picked-up image in which an image of the working space of a robot including a work object is taken is displayed in a 2D display. An operator draws a geometric element on the display surface of the display by making full use of a parametric modeling technique used in CAD (Computer Aided Design) or a similar technique with an input device such as a mouse or a touch panel while looking at the 2D display where the picked-up image is displayed. Specifically, in order to create a model of the shape of the work object, the operator adjusts the position, posture, and shape parameters of the model so that they are matched with those of the work object image on the screen, and establishes a coordinate system in which translation and rotation are applied as necessary with reference to the coordinate system of the position of the center of gravity of the work object model so that the work object can be easily grasped. By adapting the coordinate system of the end effector for grasping a work object provided at the leading end of the robot arm (herein called “tool coordinate system”) to the coordinate system established in this manner, it is possible to make the robot grasp the work object. Note that 3D positions of the points and the geometric elements drawn by the operator are measured by a range measuring device such as a laser rangefinder and a stereo camera.
Patent document 2 discloses an action teaching apparatus including a storage unit that stores a shape model of the shape of an end effector provided at the leading end of a robot arm, and a display that displays the shape of the end effector. The action teaching apparatus: (1) receives the designation of a point on the end effector by an operator on the display screen where the shape of the end effector is displayed; (2) calculates a 3D position of the designated point; (3) receives an input of a posture change centered on the designated point by the operator; and (4) adjusts the posture of the end effector based on the 3D position of the designated point and the received posture change. That is, the action teaching apparatus of Patent document 2 is used to support the operator's work to input a posture change of the end effector whose position and shape are already known by graphic display. However, the action teaching apparatus of Patent document 2 cannot be used to teach operations for a work object whose shape and 3D position are unknown to the robot.
Patent document 3 discloses an action teaching apparatus to teach a work point of a work object whose shape and 3D position are already known to a robot by using graphic display. For example, if the work object is a car body, the action teaching apparatus is used to teach welding work points on the body. The action teaching apparatus of Patent document 3 also includes a storage device in addition to the graphic display. The storage device stores shape data of a work object obtained by polyhedron approximation based on a world coordinate system, i.e., coordinate system common to both the robot and the work object. The action teaching apparatus accesses the storage device, displays the shape of the work object in two dimensions in the graphic display, and receives designation of a work surface and a work point of the work object by an operator on the display screen. Then, the action teaching apparatus superimposes and displays the posture of the robot on the graphic display. Further, after receiving correction for the posture of the robot by the operator, the action teaching apparatus determines the posture of the robot, transforms the determined posture onto the robot coordinate system, and teaches it to the robot.
[Patent Document 1]
    Japanese Unexamined Patent Application Publication No. 2003-256025[Patent Document 2]    Japanese Unexamined Patent Application Publication No. 2005-111618[Patent Document 3]    Japanese Unexamined Patent Application Publication No. 1-283603[Patent Document 4]    Japanese Unexamined Patent Application Publication No. 2004-333422